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"But Farmer Hoggett knew that little ideas 
that tickled and nagged and refused to go away 

should never be ignored 
for within them lie the seeds of destiny." 

-from the movie Babe based on the novel 
of the same title written by Dick King-Smith 
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Cox’s Contribution 


Cox generalized 
implication among 
logical statements to 
degrees of implication 
represented by real 
numbers. 
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Associativity of the Conjunction 


Consistency with associativity of the conjunction 

p(b a c a d | a) = p((b a c) a d \ a) 

= p(bA(cAd)\a ) 


Results in the Product Rule 

p(b Ac\a) = p(b | a) p{c \ a a b) 


5 August 2003 


MaxEnt 2003 





Complementation 


Consistency with complementatation 

p(b | a) = p(~ (~ b ) | a) 


Results in the Sum Rule 

p(b | a) + /?(~ b\a) = \ 
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Commutativy of the Conjunction 


Consistency with the commutativity of the conjunction 

p(a Ab\h) = p(b Aa\h) 


Results in Bayes Theorem 


n(a \ h a h\ - 

\a a h) p(a h) 

y 6/ | U / \ # L j 

p(b h) 
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Inferential Calculus 


The inferential calculus (probability theory) derives directly 
from consistency with 

Associativity 

Commutativity 

Complementation 

These are BAT mathematical ideas, 
not restricted to inference. 


5 August 2003 


MaxEnt 2003 




Other Trails 
Up the Mountain 



Other Trails up the Mountain 


Dr. Aczel’s major contribution has been his thorough 
investigation of the functional equations central to this 
development. 

Ray Smith and Gary Erickson investigated all possible forms of 
the associativity equations. 

Anton Garrett derived the sum and product rules using 
consistency with the NAND operation. 


Certainly I am missing other contributions... 
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A More General Derivation 


Ariel Caticha derived the sum rule from consistency with 

associativity of the disjunction 

p(avb\h) = p(a \ h) + p(b \ h) 

when a and b are logically independent. 


The product rule can then be derived from consistency with 
distributivity. 

p(aAb\h) = p(a\b a h) p(b\h) 
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What is the Big Deal? 


Because the Sum and Product Rules are 
not JUST associated with 
Boolean algebras. 


They are 
associated with 
Distributive Algebras! 
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Order 




Ordering Relations 


To sets of objects, one can often impose additional structure, 
such as a binary ordering relation denoted by a < b, which 

satisfies for all a, b, c (Birkhoff 1967): 


Now a < b is read “b contains a” or “b includes a” 

If a < b and a * b one can write a < b and read 
“ a is less than b" or “a is properly contained in b” . 

If a < b but a < x <b is not true for any x in the poset P, then 
we say that “b covers a”, written a -< b. 


PI. For all a, a < a . 

P2. If a < b and b <a, then a = b 

P3. If a < b and b < c , then a <> c 


(Reflexive) 

(Antisymmetry) 

(Transitivity) 
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Posets 


Together a set and an ordering relation are called a partially 
ordered set, or a poset. 

The set of Natural numbers {1 , 2, 3, 4, 5} with the binary 
ordering relation “is less than or equal to” < is a poset. 

It is clear that: 

2 < 2 

As 2 ^ 2 and 2^2, then 2 = 2 
As 2 ^ 3 and 3 < 4 , then 2^4 

Also 2 < 3 as 2 < 3 but 2*3 

And 2^3 as 2 <3 but there is no Natural number x such 
that 2 < x < 3 


(Reflexive) 

(Antisymmetry) 

(Transitivity) 
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Visualizing the Structure 


The covering relation can be used to visualize the structure of 
a poset. 


5 

Whenever a < b draw b above a\ 

4 

3 

2 

1 
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Haase Diagrams 


The covering relation can be used to visualize the structure of 
a poset. 


5 

Whenever a < b draw b above a. 

4 

And whenever a < b connect the 
elements with a line: 3 

This poset forms a chain. 2 

1 
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Incomparable Elements 


There are times where for a given ordering relation, it is not 
true that a < b or b < a . 


We then write a\\ b read “a is incomparable to b” 

Perhaps for the ordering relation “is healthier than’’ we have 
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Antichains 


The diagram corresponding to a poset of three incomparable 
elements is a picture with the elements placed side-by-side. 





This is called an antichain. 
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A More Useful Illustration 


Consider the powerset of the set S = { a, b, c } 


This is the set of all possible subsets of S: 

P(S) = {0, {a}, {b}, {c}, {a,b}, {a,c}, {b,c}, {a,b,c} } 


A natural ordering is the relation “is a subset of’, — 

P = ({0, { a }, {b}, {c}, {a,b}, { a,c }, {a,b,c} } c) 
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The First Level 


P = ({ 0 , { a }, {b}, {c}, {a,b}, { a,c }, {l b,c }, {a,b,c} } c) 


First we note that 0 £= W , 

from which we also see that: 0 ^ } 


So we draw { a } above 0 
and connect them with a line. 

0 


w 
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Completing the First Level 


P = ({0, { a }, {b}, {c}, {a,b}, { a,c }, {l b,c }, {a,b,c} } c) 


It is also true that 0 -< {b} and 0 -< {c} 
so we draw them above 0 as well and 
connect them with lines. 

However, {a} || { b} as neither one is the 
subset of the other. 


In addition, {a} || {c} and {b} || {c} . 

So we draw them on the same level and 
do not connect them. 


{a} {b} {c} 

\I0 



5 August 2003 


MaxEnt 2003 




The Second Level 


P = ({ 0 , { a }, {b}, {c}, {a,b}, { a,c }, {l b,c }, {a,b,c} } c) 


Now we note that {^} is covered by 
two elements {a,b} and {a, c\ 


{a, b} { a , c} 

M ib} M 

\I0 

0 
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The Second Level 


P = ({ 0 , { a }, {b}, {c}, {a,b}, { a,c }, {l b,c }, {a,b,c} } c) 


These elements also 
cover {^} and { c } 


{a, b } {a, c } 

IX \ 

{a} {b} M 

\I0 

0 
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Completing the Second Level 


P = ({0, { a }, {b}, {c}, {a,b}, { a,c }, {l b,c }, {a,b,c} } c) 


Now {b, c } also 
covers W\ and {^1, 
but these top elements 
are also incomparable. 


{a, b } {a, c } {b, c } 

IX X] 

{a} {b} M 

\I0 

0 
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The Third Level 


P = ({0, { a }, {b}, {c}, {a,b}, { a,c }, {l b,c }, {a,b,c} } c) 


Finally i a -> b, c } 
covers all three two- 
element subsets. 


{a, b, c} 

{a, b} {a, c} {b, c } 

IX XI 

{a} {b} M 

XIX 

0 
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The Powerset of {a, b, c} 


P = ({ 0 , { a }, {b}, {c}, {a,b}, { a,c }, {l b,c }, {a,b,c} } c) 


{a, b, c } 



{a, b} {a, c } {b, c} 


IXXI 


{a} {6} {c} 

\I0 




Is a subset of 
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Lattices 



Lattices 


A lattice is a poset P where every pair of elements x and y has 
a least upper bound called the join x v y 


a greatest lower bound called the meet x a y 


The green 
elements are 
upper bounds 
of the blue circled 
pair. The green 
circled element is their 
least upper bound or 
their join. 


{a, b, c} 


{a, b} {a, c} 



{b} V {c} = {b, C} 


0 


Similarly 

{a, b} a {b, c} = {6} 


5 August 2003 


MaxEnt 2003 




Lattice Identities 


The Lattice Identities 

LI. im = i, jcvi = j Idempotent 

L2. x /\ y = y /\ x, xv y = y v x Commutative 

L3. x a (y a z) = (x a y) a z, xv (y v z) = (x v y) v z Associative 
L4. x a (xv y) = x v (x a 7) = x Absorption 

If x< y the meet and join follow the Consistency Relations 
Cl . x a y = x (x is the greatest lower bound of x and y) 

C2 . xv y = y (y is the least upper bound of x and y) 
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The Dual Lattice 


The dual lattice can be obtained by reversing the ordering relation 


L 


{a, b, c } 




0 


L 


d 


0 


{a,b} {a, c} {b, c} > K 

IX XI <=> 

{a} {b} {c} 



W {b} {c} 

IXX 

{a, b } {a, c } {b, c} 




{a, b, c} 


This flips the lattice upside-down and exchanges meets and joins. 
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Top and Bottom Elements 


The greatest element is called the top 
and is symbolized by 1 , or T. 

So that T >x for all x in L. 


The east element is called the 
bottom and is symbolized by 0 or _L 
So that x > 0 for all x in L. 


L 


{a, b,c} = T 



[XX 


{«} { b } {c} 

0 = 1 
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Distributive Lattices 


A »is ri u ive Lattice possesses structure additional to LI -4. 
It also satisfies the following identity for all elements x, y, z. 

x a ( y v z) = (x a v) v (x a z) 

D1. Distributive 

x v (y a z) = (x v y) a (x v z) 

Note that these two equations are related by duality as the 
dual of a distributive lattice is a distributive lattice. 
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Complemented Distributive Lattices 


There is a special case of a distributive lattice that possesses 
an interesting property where each element is associated with 
one other element called its complement . 

The complement has these properties 

B1. xa~x = 1 xv~x = T 
B2. ~ (~ x) = x 

B3. ~(xAy) = ~xv~y ~(xv.y) = ~xA~j 
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Lattices and Algebras 


Associated with every lattice is an algebra. 


Thus a lattice can be expressed either in terms of its elements 


and its ordering relation 



or in terms of its algebra 



or perhaps 
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Implication as an Ordering Relation 


At this point the algebra associated with the complemented 
distributive lattice should look familiar - Boolean algebra. 
More commonly, the poset is a set of assertions and the 
ordering relation is “implies” ^ = 


a l) c 

W/ 


T is the ruism 
_L is the Absurdity 
v s Logical Disjunction 
a 3 Logical Conjunction 


ay by c 


A 



ay b aye b 

IX X] 


v c 
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0 





Boolean Lattices 






ay by c 



ay b aye by 


OCX] 

a b c 



The elements that cover 0 are called atoms. 0 
In a Boolean lattice the atoms are the mutually exclusive 
assertions. All other elements are joins of the atoms. 
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Join-Irreducible Elements 



a v 


ay by c 



b aye by 




a 



c 



An important subset of a lattice 
is the set of elements that cannot 
be written as a join of elements. 
They are the join-irreducible 
elemei 

In a Boolean lattice, these are the 
atoms, which by themselves form 

an antichain. 

One can use this property to 
identify a Boolean lattice. 
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Powersets Revisited 


{a, b, c } 



OCX 


M {6} M 

\l/ 

0 


The powerset (the set of all 
subsets) of a set forms a 
Boolean lattice under the 
ordering relation 

Note that the atoms form an 
antichain. 

The complement of a set S is 
the set T \ S. 
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Deductive Inference 


Deductive inference is easy. 

For all x * y 

If x < y then x — » y. 

If x < y then y 7^ x. 


a v 


ay by c 



b aye by 


[XX 

a c 


If X 


y then x tX and y^x. 




Note that: 0 

The absurdity -L ( 0 ) implies everything. 

The truism T is implied by everything. 
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Inductive Inference 


Deduction is nice and all, but 
sometimes I know that one of a 
set of possibilities is true 


ay by c 



And I want to know to what 
degree that knowledge implies a 
more simple hypothesis. 

But since ay by c > a 


ay b aye by 



it is only true that a — > ay by C 



Not vice versa! 
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Generalizing Implication 


To generalize to degrees of implication, we introduce a real- 
valued function that takes two lattice elements to a real 
number: 


p:LxL^m 


ay by c 


In particular we write this as: 

p(a | T) 

Since one of a, b, or c is true, the 
truism can be considered to be 
our prior information, in part. 



[XX 

a b c 


0 
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Other Views 


This function can be written in two different ways. 


Typically when the premise is the truism, we can write this 
function as a function that takes a single lattice element to a 
real number: , x 

P( a ) ay by c 


And write the function as 



p(a\h) 


in other cases. 


ay b aye bye 


IX X 

a b c 



0 
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Following the Rules 


To be consistent with the Boolean lattice structure, this new 
measure must follow the rules: 

LI-4, D1, and B1-3. 

The key ones we use are 


L3 

Associativity of a 

X A (y A z) = (x A y) A z 

D1 

Distributivity 

x v (y a z) = (x v y) a (y v z) 

L2 

Commutativity of a 

x A y = y A x 
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Valuations on Lattices 


A valuation on a lattice is defined as a function that takes 
a lattice element to a commutative ring. 

v : L — > A where A is an element of a 

commutative ring. 


This has been investigated in 
great detail by a small group 
of mathematicians led by 
Gian-Carlo Rota. 
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Probability as a Valuation 


The function we defined for probability is a valuation. 


v : L l —> * A 
p:LxL^di 

P(a | T) = p{a) 


ay by c 

XX 

ay b aye bye 


[XX 

a b c 




5 August 2003 


MaxEnt 2003 




Why is this Useful? 


There is a theorem that all valuations can be uniquely 
determined from the valuations on the join-irreducible 
elements of the lattice 


AND 

their assignments are arbitrary! 

Thus, by assigning the 
prior probabilities 

p(a) p(b ) p(c) 

the probabilities of any other pair of 
elements in the lattice is determined. 


a v b v c 



ay b aye by 
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Assigning Priors is Hard 


One can now see why assigning priors is difficult. 


There is NO structure in the Boolean algebra of assertions that 
can guide us in these assignments. 


We must employ other principles 
to assign them. 


ay by c 



ay b aye bye 
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Prior Probabilities 


Symmetry, constraints and consistency 
with other aspects of the problem can be 
used to assign prior probabilities. 

Order-theoretic principles dictate the 
remaining probabilities. 
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Probability Theory 

from 



Probability Theory and Physics 


Thanks to the efforts of Ed Jaynes, Myron 
Tribus and others, I am able to wave my 
hands and say that I can derive much of 
physics from order-theoretic principles. 

Understanding Maximum Entropy 
remains a challenge. 
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Geometry 
\ from 



Geometric Probability 


Many geometric laws can be derived from order-theoretic 
considerations. 

Geometric objects can 
be ordered, conjoined 
and disjoined often 
resulting in a distributive 
lattice structure. 

Valuations are assigned, 
which are invariant with 
respect to Euclidean 
translations and rotations. 
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These Valuations have a Basis! 


For three-dimensional Euclidean geometry all invariant 
valuations can be written as a linear combination of 4 basis 
valuations. 


V = volume 
A = surface area 
W = mean width 



X = Euler characteristic 
= aV + bA + cW + dx 
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Euler Characteristic 


The Euler characteristic is a valuation. 

X=F-E+V 


For a 3D tetrahedron it is found by 

X(tetra ) = 4 - 6 + 4 = 2 
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Euler Characteristic 


For a cube, it is 

X = F - E + V 
X(cube) = 6-12 + 8 
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Corresponding Lattice 
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Lattice Structure of a Cube 


cube 



rank 3 
rank 2 
rank 1 
rank 0 
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Mobius Functions 


A Mobius function for a partially ordered set P is a 
function that satisfies: 


jl(x,x ) = 1, 

xEP 

V y(x,z) = 0 

x<y 

x<z<y 


/u(x,y) = 0 

y>x 


These functions are important for inverting other 
functions on the lattice, as we will see later. 

This is also related to the Euler characteristic in 
distributive lattices. 
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Inclusion-Exclusion Principle 


Mobius functions allow us to compute valuations on elements 
higher in the lattice based on linear combinations of valuations 
of elements lower in the lattice. 

For many familiar structures this leads to 

Rota’s inclusion-exclusion principle where when summing 

we add at one level and subtract at the next and so on. 

We saw this with the Euler characteristic 

X = F - E + V 


And we’ll see it also on the next slide... 
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Joining Parallelotopes 


P^P 2 



v(P t V P 2 ) = v(/J) + v(P 2 ) - v(P l A p 2 ) 
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Ariel Caticha 


Ariel has also developed a very 
interesting derivation of quantum 
mechanics using Cox’s method 
applied to experimental setups 

rather than logical statements. 

The concept of consistency with 
the order-theoretic structure is 
central here as well. 
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Particles and Motion 


A particle moves from Xj to x f 



▲ 


time 
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A Little More Complex 


A particle moves from Xj to x 1 
and then from x 1 to x f 

x. 



▲ 


time 
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A Little More Complex 


A particle goes from Xj to x f 
via x 1 or x\ 


[x r , (*! , *! ), x,. ] 




▲ 


time 
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Experimental Setups 


We can look at this v 

experimental setup as - 



time 
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The Meet Operation 


We can look at this 
experimental setup as 

being a combination of 
two setups... 

[x f ,x t ] A 

= [x f ,x l ,x i ] 



▲ 



time 
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The Join Operation 



▲ 


This is a different way to 
combine setups 


[X f , JCj , x t ] v , x i ] 
= [XpCx,,*,'),*,.] 




time 
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Associativity of the Meet 


The meet is associative, as 


[x.,x 2 ] a [x 2 ,xj a [x, , x z . ] 


= [x /? x 2 ] A ([x 2 ,x 1 ] A [x 1? xj) 
= ([x,,x 2 ] A [x 2 ,xJ)a [x, , x. ] 

^Xy •, X 2 •, Xj ■, X. j 
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Meet NOT Commutative! 


However, experimental 
setups are not commutative 
under the meet operation! 


This is because it makes 
no sense for a particle to 
go from x 1 to x f and then 
from Xj to x 1 ! 





▲ 


time 
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Associativity of the Join 


The join is also associative, as 


[Xj , x", X- ] V [x f j Xj •) JCj j V ^X y- •) j JCj j 

= i >/,*n v C*/>*n v ix,*j) 

= ([XpX,"] v [Xy,x[])v [x v x t ] 


= [x f ,{x[,x[,x i),x,] 


As long as each of these joins is allowed 
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Join is Commutative 


Joins are commutative 


[x f , x", x t ] v [x f ,x[,x t ] = [x f ,x[,x j ] v [x f ,x",x t ] 


7 


/ 


7 
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NOT a Lattice Structure 


What is interesting about setups is that because not 
all meets and joins exist, setups do not form a lattice 
structure. 

They do form a poset however. 

As the measure we will define is not probability, Ariel 
represented it with^(a) rather than p(a \ i ) 

So lets continue... 
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Sum and Product Rules Again 


Caticha showed that the 

Sum Rule is derived from Associativity of the Join. 

\p(av b) = \p ( a ) + tp ( b ) 

Product Rule from Distributivity. 

ip (a a b) = ip (a) ip ( b ) 
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Amplitudes 


If we let the valuations on this poset take on complex 
values, we have quantum mechanical amplitudes. 


Caticha showed that one can then easily derive 

Schrodinger’s Equation. 

Feynman Path Integrals are simply analogous to 
marginalizations. 


5 August 2003 


MaxEnt 2003 




Probability and QM 


Quantum Mechanics is NOT Probability Theory 

Probability is a degree of implication defined on a 
partially ordered set of logical statements. 

Amplitudes are an analogous measure defined on 
the partially ordered set of experimental setups. 

This is exciting as it suggests that other analogous 
measures can be constructed for other partially 
ordered sets, leading to new laws! 
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Take Home Message 



Symmetry + Order = Laws 
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Carlos Rodriguez for their discussions, 
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I would like to thank Robert Fry for introducing 
me to this fascinating area of study. 
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Cox Details 


Conjunctions 


We look at the conjunction of two assertions implied by a premise 

(a ^ b ac) 

and take as an axiom that this is a function of 

( a — > b) and (ci A b —■ > c) 

so that 

(a b a c) = F&a -&),(« a* - c)] 
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Conjunctions 


We now conjoin an additional assertion 
{a — 3 *1 ~> / \ c :Ai d ) I = (a — > (Z ? a c ) a d) 


= F\(a — > b a c), (a a b a c — > rf)l 


Letting 


x = (a —* b) y = (a a b c) z = (a hb ac d) 


We have 


(a ^ b A C A d) = 
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Associativity of the Conjunction 


We could have grouped the assertions differently 
(a — » b a c a d) = (a — > b a (c a d)) 

-*t(* — > Z>),(<2 A b — > C A <0] 

= F[(a -* *), F[(a a b c),(a a b a c 
= F[x, F[y,z]] 



This gives us a functional equation 

f[f[x, > ]_-]= F [ x , f[v , -]] 


5 August 2003 


MaxEnt 2003 




The Product Rule 


Functional Equation 

> ]_-]= F[x, F[v ,r]] 


As a particular solution we can take f[x, >>]= xy 
which gives 

(a — > b a c) = (« — > &)(a a b ^ c) 
and can be written in a more familiar form by changing notation 


(b A c\ 

a) = (b\ 

a ) ( c 

a a b) 
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The Product Rule 


In general however the solution is 

F\x,y\= G-‘[g[x]g[j]] 


where G is an arbitrary function. 




We could call G probability! 
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Logical Complements 


The degree to which a premise implies a statement 
determines the degree to which it implies its contradictory. 

{a -*~&) = /[(a-*&)] 
(a-»—(~6)) = /[(fl-»~6)] 

(a b)) = /[/[(« -»•&)]] 

(a -*2>) = f[f\ia -»•&)]] 

= x 
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The Sum Rule 


Another functional equation 

/[/HI = x 


A particular solution f(x)=l-x 
is which gives 

(a —* b) + (a —* ~ b) = 1 
(b | a) + (~ b | a) = 1 

In general 

g(Z>|a) + g(~Z>|a) = C 
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Putting it T ogether 


The solution to the first functional equation puts some 
constraints on the second 

G\b a c | a\= G\b | a]g[c | a a b\ 
g\b | a]+ g[~ b\a]=C 

The final general solution is G [1 ri x r 
and we have 

(b a c | a) r = ( b | a) r (c \ a a b) r 
(b | a) r + (~ Z) | a) r = C 
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The Rules of Probability 


Setting r = C = 1 and writing the function g(x) = G(x) as p(x) 
we recover the familiar sum and product rules of probability 

p(b Ac\a) = p(b | a) p(c \ a a b) 
p(b | a) + p(~ b | a) = 1 


Note that probability is necessarily conditional! 

And we never needed the concept of frequencies of events! 

The utility of this formalism becomes readily apparent when the 
implicant is an assertion representing a premise and the implicate 
is an assertion or proposition representing a hypothesis 

pihypothesis \ premise ) = ( premise — > hypothesis ) 
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Commutativy of the Conjunction 


The symmetry of the conjunction of assertions 

a a b = b a a 

means that under implication 

(h — > a a b) = (h — > b a a) 

also written as 

p{a a b | h) = p(b Aa\h) 

which means we can write 

p(a Ab\h) = p(a \ b a h) p(b | h) 

= p(b | a a h) p(a \ h ) 
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Bayes’ Theorem 


p(model | data, /) 

Posterior Probability 


Likelihood 
Prior Probability Evidence 
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Bayes’ Theorem 


Bayes’ Theorem is a Learning Rule 


Prior Knowledge 



pimodel | data, /) = p{model\I ) 


/ 


Improved State 
of Knowledge 



Data Dependent Term 
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Inferential Calculus 


In short we have the following calculus: 

Product Rule 

P(x A y | /) = p(y | x A /) /?(x | /) associativity of a 


Sum Rule 

p{x 1 1) + /?(~ x | /) = 1 complements x = ~ (~ x) 


Bayes Theorem 

PC' I x a I) = p(y 1 1) p ( X j y A l ^ 

p(x 1 1) 


commutativity of a 
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